Combining effective images in electronic device having a plurality of cameras

ABSTRACT

An image combining method in an electronic device having a plurality of cameras. In response to a photographing signal, images successively photographed through at least a first camera are successively stored. A first image is selected from the successively photographed images which satisfies a predetermined classification reference, such as a degree of blurring, a facial expression, and/or a shooting composition. A second image is captured through a second camera; and the first and second images are then combined. The combined image may be a picture-in-picture (PIP) type combination image. The first and second cameras may be front and rear cameras of a portable terminal, or vice versa. The successive image capture and selection technique may also be applied to the second camera.

CLAIM OF PRIORITY

This application claims the priority under 35 U.S.C. §119(a) to KoreanApplication Serial No. 10-2013-0018761, which was filed in the KoreanIntellectual Property Office on Feb. 21, 2013, the entire content ofwhich is hereby incorporated by reference.

BACKGROUND

1. Technical Field

The present disclosure relates generally an electronic device having aplurality of cameras and more particularly, to a method for combiningeffective images among images photographed by using a plurality of suchcameras.

2. Description of the Related Art

Currently, with the rapid development of communication technologies,functions of portable electronic devices such as portable terminals havebeen expanded, and accordingly, more diverse User Interfaces (UIs) andvarious functions using the UIs have been provided. In order to increasean effective value of portable terminals and meet various user demands,a variety of portable terminal applications have been developed.

Application functions using a camera have expanded greatly. For example,a camera phone function as well as a video phone function with videoconferencing are typical applications in today's smart phones. Anotherexample is an image combining application which is downloadable througha wireless Internet service, in which various image combiningapplications are possible, e.g., combining a user's hair style in apicture photographed by the user.

Further, in order to satisfy various tastes of users, a dual cameraportable terminal with front and rear cameras is currently available. Adual camera portable terminal can display a main image photographedthrough one camera on a preview screen, while a sub image photographedthrough the other camera is overlaid with the main image in a form ofPicture In Picture (PIP). In the portable terminal having the dualcamera, a rear camera of the portable terminal is fixed to face a reardirection of the portable terminal, and a front camera of the portableterminal is fixed to face a front direction so as to photograph theuser.

However, while it's possible to simultaneously identify two imagesthrough the two cameras facing in opposite directions, severallimitations with current technology are apparent. First, in the subimage displayed smaller than the main image in the PIP form, the userhas a difficulty in identifying a quality of the sub image with thenaked eye due to a size limitation of the sub image. In addition, whendifferent images photographed through the actual dual camera arecombined, a combined image may be unsatisfactory if hand tremblingoccurs.

Thus, the conventional art does not provide a method of guaranteeingquality of the combined image in a dual camera portable terminal.

SUMMARY

Embodiments of the present invention combine two images which satisfy apredetermined standard among images photographed simultaneously or withtime differences therebetween by using a plurality of cameras in orderto guarantee a quality of the combined image.

A method of combining effective images in an electronic device having aplurality of cameras is provided. In response to a photographing signal,images successively photographed through at least a first camera aresuccessively stored. A first image is selected from the successivelyphotographed images which satisfies a predetermined classificationreference, such as a degree of blurring, a facial expression, and/or ashooting composition. A second image is captured through a secondcamera; and the first and second images are combined.

In various embodiments, the combined image may be a picture-in-picture(PIP) type combination image. The first and second cameras may be frontand rear cameras of a portable terminal, or vice versa. The successiveimage capture and selection technique may also be applied to the secondcamera.

In an embodiment, an electronic device having a plurality of cameras forcombining effective images is provided. The electronic device includes afirst camera which successively photographs images in response to aphotographing signal. A second camera photographs a second image; and acontroller selects a first image satisfying a predeterminedclassification reference from the images successively photographedthrough the first camera and combines the first and second images.

In another embodiment, a method of combining effective images in anelectronic device having front and rear cameras is provided. In responseto a photographing signal, front and rear images are captured from thefront and rear cameras, respectively. A difference is determined in atleast one of brightness and tone between the front and rear images. Atleast one of the front and rear images is modified to equalizebrightness or tone between the front and rear images if the differenceexceeds a predetermined threshold. The front and rear images are thencombined.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating the inside of a portable terminalaccording to an embodiment of the present invention;

FIG. 2 is a front perspective view of a portable terminal according toan embodiment of the present invention;

FIG. 3 is a rear perspective view of a portable terminal according to anembodiment of the present invention;

FIG. 4 is an operation flowchart illustrating an image combining processaccording to a first embodiment of the present invention;

FIG. 5 illustrates an example for describing an image combining methodof FIG. 4;

FIGS. 6A and 6B illustrate an example for describing another imagecombining method of FIG. 4;

FIG. 7 is an operation flowchart illustrating an image combining processaccording to a second embodiment of the present invention;

FIG. 8 illustrates an example for describing an image combining methodof FIG. 7;

FIG. 9 is an operation flowchart illustrating an image combining processaccording to a third embodiment of the present invention;

FIG. 10 illustrates an example for describing an image combining methodof FIG. 9;

FIG. 11 is an operation flowchart illustrating an image combiningprocess according to a fourth embodiment of the present invention; and

FIG. 12 illustrates an example for describing an image combining methodof FIG. 11.

DETAILED DESCRIPTION

Hereinafter, various embodiments of the present invention will bedescribed with reference to the accompanying drawings. However, thepresent invention is not limited or restricted by the embodiments of thepresent invention described herein. The same or like elements will bedesignated by the same or like reference numerals although they areshown in different drawings.

While terms including ordinal numbers, such as “first” and “second,”etc., may be used to describe various components, such components arenot limited by the above terms. The above terms are used only todistinguish one component from another. For example, a first componentmay be referred to as a second component without departing from thescope of the present invention, and likewise a second component may bereferred to as a first component. The terms used herein are merely usedto describe specific embodiments, and are not intended to limit thepresent invention. An expression used in the singular encompasses theexpression of the plural, unless it has a clearly different meaning inthe context.

Embodiments discussed below are described as examples of a portable (orhand-held) terminal, but an image combining method according to thepresent invention can be applied to various portable and fixedelectronic devices such as a camera, a navigation device, a television(TV), an Automatic Teller Machine (ATM) of a bank, a Point of Sale (POS)device of a store and the like as well as terminals such as a mobilephone, a smart phone, a tablet PC and the like. Further, an electronicdevice according to the present invention can be a flexible device or aflexible display device. A representative configuration of the portableterminal described below corresponds to a configuration of the mobilephone. Some components of the representative configuration of theportable terminal may be omitted or changed as necessary.

The present disclosure provides an image combining method in anelectronic device. The image combining method includes processes ofselecting images one by one from a plurality of images successivelyphotographed by a plurality of cameras through application of apredetermined classification reference, combining the selected images,and storing the combined images. The predetermined classificationreference refers to a numerical value of a photography metric, examplesof which include but are not limited to a blur amount within an image, afacial expression of a person, and a shooting composition. Accordingly,the combined image is generated by combining the images satisfying thepredetermined classification reference, which causes an effect ofimproving a quality of the combined image.

FIG. 1 is a block diagram schematically illustrating a portableterminal, 100, according to an embodiment of the present invention.Portable terminal 100 can be connected with an external electronicdevice (not shown) by using at least one of a communication module 120,a connector 165, and an earphone connecting jack 167. Examples of theexternal electronic device include any of various devices attached to ordetached from the portable terminal through a cable, such as anearphone, an external speaker, a Universal Serial Bus (USB) memory, acharger, a cradle/dock, a DMB antenna, a mobile payment related device,a health management device (blood sugar tester or the like), a gamemachine, a car navigation device and the like. Further, the externalelectronic device can be a Bluetooth communication device, a Near FieldCommunication (NFC) device, a WiFi Direct communication device, and awireless Access Point (AP) which can be wirelessly connected. Inaddition, the portable terminal 100 can be connected with anotherportable terminal or portable electronic device, for example, a mobilephone, a smart phone, a tablet PC, a desktop PC, or a server wirelesslyor through a wire.

Portable terminal 100 includes a touch screen 190, at least one touchscreen controller 195, a controller 110, a communication module 120, amultimedia module 140, a camera module 150, an input/output module 160,a sensor module 170, a storage unit 175, and a power supplier 180.

The communication module 120 includes a mobile communication module 121,a sub communication module 130, and a broadcasting communication module141.

The sub communication module 130 includes at least one of a wireless LANmodule 131 and a near field communication module 132, and the multimediamodule 140 includes at least one of an audio reproduction module 142 anda video reproduction module 143.

The camera module 150 includes a first camera 151 and a second camera152, described further below. The input/output module 160 includes atleast one of a button(s) 161, a microphone 162, a speaker 163, avibration device 164, a connector 165, and a keypad 166.

The controller 110 includes a CPU 111, a ROM 112 storing a controlprogram for controlling the portable terminal 100, and a RAM 113 used asa storage area for storing a signal or data input from the outside ofthe portable terminal 100 or for work performed in the portable 100. TheCPU 111 can be a single core, a dual core, a triple core, or a quadcore. The CPU 111, the ROM 112, and the RAM 113 can be mutuallyconnected to each other through an internal bus.

Further, the controller 110 can control the communication module 120,the multimedia module 140, the camera module 150, the input/outputmodule 160, the sensor module 170, the storage unit 175, the powersupplier 180, the touch screen 190, and the touch screen controller 195.

The controller 110 detects a user input as an input unit 168 or a touchinput means such as a user's finger touches or approaches one object oris located close to the object in a state where a plurality of objectsor items are displayed on the touch screen 190, and identifies an objectcorresponding to a position of the touch screen 190 where the user inputis generated. The user input through the touch screen 190 includes oneof a direct touch input of directly touching the object and a hoveringinput which is an indirect touch input of approaching the object withina preset recognition range but not directly touching the object. Forexample, when the input unit 168 is located close to the touch screen190, an object located directly below the input unit 168 may beselected. According to the present invention, user inputs include agesture input through the camera module 150, a switch/button inputthrough the button 161 or the keypad 166, a voice input through themicrophone 162, and the like as well as the user input through the touchscreen 190.

The object or item (or function item) is displayed on the touch screen190 of the portable terminal 100. For example, the object or itemindicates at least one of an application, a menu, a document, a widget,a picture, a video, an e-mail, an SMS message, and an MMS message, andcan be selected, executed, deleted, canceled, stored, and changed by auser input means. “Item” as used herein can mean a button, an icon (orshort-cut icon), a thumbnail image, and a folder storing at least oneobject in the portable terminal. Further, the item can be displayed in aform of an image, a text or the like.

The short-cut icon is an image displayed on the touch screen 190 of theportable terminal 100 to rapidly execute each application or operationof phone communication, a contact number, a menu and the like basicallyprovided in the portable terminal 100. When a command or selection forexecuting the application or the operation is input, the short-cut iconexecutes the corresponding application.

Further, the controller 110 can detect a user input event such as ahovering event as the input unit 168 approaches the touch screen 190 oris located close to the touch screen 190.

When a user input event is generated according to a preset item or in apreset manner, the controller 110 performs a preset program actioncorresponding to the user input event.

The controller 110 can output a control signal to the input unit 168 orthe vibration device 164. The control signal includes information on avibration pattern and the input unit 168 or the vibration device 164generates a vibration according to the vibration pattern. Theinformation on the vibration pattern indicates the vibration patternitself, an identifier of the vibration pattern or the like.Alternatively, the control signal simply includes only a request forgeneration of the vibration.

The portable terminal 100 includes at least one of the mobilecommunication module 121, the wireless LAN module 131, and the nearfield communication module 132.

The mobile communication module 121 enables the portable terminal 100 tobe connected with the external device through mobile communication byusing one antenna or a plurality of antennas according to a control ofthe controller 110. The mobile communication module 121transmits/receives a wireless signal for voice phone communication,video phone communication, a Short Message Service (SMS), or aMultimedia Message Service (MMS) to/from a mobile phone (not shown), asmart phone (not shown), a tablet PC, or another device (not shown)having a phone number input into the portable terminal 100.

The sub communication module 130 may include at least one of thewireless LAN module 131 and the near field communication module 132. Forexample, the sub communication module 130 may include only the wirelessLAN module 131, only the near field communication module 132, or boththe wireless LAN module 131 and the near field communication module 132.

The wireless LAN module 131 can be Internet-connected according to acontrol of the controller 110 in a place where a wireless Access Point(AP) (not shown) is installed. The wireless LAN module 131 supports awireless LAN standard (IEEE802.11x) of the Institute of Electrical andElectronics Engineers. The near field communication module 132 canwirelessly perform near field communication between the portableterminal 100 and the external device (not shown) according to a controlof the controller 110. A near field communication scheme may includeBluetooth, Infrared Data Association (IrDA) communication and the like.

The controller 110 can transmit the control signal according to a hapticpattern to the input unit 168 through the sub communication module 130.

The broadcasting communication module 141 can receive a broadcastingsignal (for example, a TV broadcasting signal, a radio broadcastingsignal, or a data broadcasting signal) and broadcasting supplementinformation (for example, Electric Program Guide: EPG or ElectricService Guide: ESG) output from a broadcasting station through abroadcasting communication antenna (not shown) according to a control ofthe controller 110.

The multimedia module 140 includes the audio reproduction module 142 orthe video reproduction module 143. The audio reproduction module 142 canreproduce a digital audio file (for example, a file having a fileextension of mp3, wma, ogg, or way) stored or received according to acontrol of the controller 110. The video reproduction module 143 canreproduce a digital video file (for example, a file having a fileextension of mpeg, mpg, mp4, avi, mov, or mkv) stored or receivedaccording to a control of the controller 110. The video reproductionmodule 143 can reproduce the digital audio file.

The multimedia module 140 may be integrated with the controller 110.

The camera module 150 includes the first and second cameras 151 and 152for photographing a still image or a video according to a control of thecontroller 110. Further, the camera module 150 includes at least one ofthe barrel 155 for performing zoom-in/zoom-out for photographing asubject, the motor 154 for controlling a motion of the barrel 155, andthe flash 153 for providing an auxiliary light source required forphotographing the subject. The first camera 151 may be disposed on afront surface of the portable terminal 100 to typically capture aclose-up image of the user. The second camera 152 may be disposed on arear (back) surface of the portable terminal 100 so as to typicallyphotograph a scene or subject as in a conventional camera. Otherpositions for the first and second cameras 151, 152 are also possible.

Each of the first and second cameras 151 and 152 includes a lens system,an image sensor and the like. Each of the first and second cameras 151and 152 converts an optical signal input (or photographed) through thelens system to an electrical image signal and outputs the convertedelectrical image signal to the controller 110. Then, the userphotographs a video or a still image through the first and secondcameras 151 and 152 by touching a predetermined shutter icon on thedisplay or pressing a physical shutter key/button if provided.

The input/output module 160 includes at least one of at least one button161, at least one microphone 162, at least one speaker 163, at least onevibration device 164, the connector 165, the keypad 166, the earphoneconnecting jack 167, and the input unit 168. Further, the input/outputmodule 160 is not limited thereto, and may be provided in order toenable a cursor control such as a mouse, a trackball, a joystick, orcursor direction keys to control a motion of a cursor on the touchscreen 190.

The button 161 may be formed on a front surface, a side surface, or aback surface of a housing of the portable terminal 100, and may includeat least one of a power/lock button, a volume button, a menu button, ahome button, and a back button.

The microphone 162 receives a voice or a sound to generate an electricalsignal according to a control of the controller 110.

The speaker 163 can output sounds corresponding to various signals ordata (for example, a wireless signal, a broadcasting signal, a digitalaudio file, a digital video file, taking a picture or the like) to theoutside of the portable terminal 100 according to a control of thecontroller 110. The speaker 163 can output a sound (for example, buttontone corresponding to phone communication, ringing tone, and a voice ofanother user) corresponding to a function performed by the portableterminal 100. One speaker 163 or a plurality of speakers 163 may beformed on a suitable position or positions of the housing of theportable terminal 100.

The vibration device 164 can convert an electrical signal to amechanical vibration according to a control of the controller 110. Forexample, when the portable terminal 100 in a vibration mode receivesvoice phone communication or video phone communication from anotherdevice (not shown), the vibration device 14 is operated. One vibrationdevice 164 or a plurality of vibration devices 164 may be formed withinthe housing of the portable terminal 100. The vibration device 164 canoperate in accordance with a user input through the touch screen 190.

The connector 165 may be used as an interface for connecting theportable terminal 100 with an external electronic device or a powersource (not shown). The controller 110 can transmit data stored in thestorage unit 175 of the portable terminal 100 to the external electronicdevice or receive the data from the external electronic device through awired cable connected to the connector 165. The portable terminal 100can receive power from the power source through the wired cableconnected to the connector 165 or charge a battery (not shown) by usingthe power source.

The keypad 166 can receive a key input from the user for the control ofthe portable terminal 100. The keypad 166 includes a physical keypad(not shown) formed in the portable terminal 100 or a virtual keypad (notshown) displayed on the display unit 190. The physical keypad (notshown) formed in the portable terminal 100 may be excluded according tocapability or a structure of the portable terminal 100.

An earphone (not shown) is inserted into the earphone connecting jack167 to be connected with the portable terminal 100.

The input unit 168 may be inserted into the portable terminal 100 forstorage when not in use, and may be withdrawn or separated from theportable terminal 100 when it is used. An attachment/detachmentrecognition switch 169 operating in accordance with attachment ordetachment of the input unit 168 is located in one area within theportable terminal 100 into which the input unit 168 is inserted, and canoutput signals corresponding to the attachment and the detachment of theinput unit 168 to the controller 110. The attachment/detachmentrecognition switch 169 may be directly or indirectly connected with theinput unit 168 when the input unit 168 is mounted. Accordingly, theattachment/detachment recognition switch 169 generates a signalcorresponding to the attachment or the detachment (that is, a signalnotifying of the attachment or the detachment of the input unit 168)based on whether the attachment/detachment recognition switch 169 isconnected with the input unit 168 and then outputs the generated signalto the controller 110.

The sensor module 170 includes at least one sensor for detecting a stateof the portable terminal 100. For example, the sensor module 170includes at least one of a proximity sensor for detecting whether theuser approaches the portable terminal 100, a luminance sensor (notshown) for detecting an amount of ambient light of the portable terminal100, or a motion sensor (not shown) for detecting motions of theportable terminal 100 (for example, a rotation of the portable terminal100, and an acceleration or vibration of the portable terminal 100), ageo-magnetic sensor for detecting a point of the compass of the portableterminal by using the Earth's magnetic field, a gravity sensor fordetecting a gravity action direction, an altimeter for measuring anatmospheric pressure to detect an altitude and a GPS module 157.

The GPS module 157 can receive radio waves from a plurality of GPSsatellites (not shown) in Earth's orbit and calculate a position of theportable terminal 100 by using Time of Arrival from the GPS satellitesto the portable terminal 100.

The storage unit 175 can store a signal or data input/output accordingto the operation of the communication module 120, the multimedia module140, the camera module 150, the input/output module 160, the sensormodule 170, or the touch screen 190. The storage unit 175 can store acontrol program and applications for controlling the portable terminal100 or the controller 110.

The term “storage unit” refers to a random data storage device such asthe storage unit 175, the ROM 112 and the RAM 113 within the controller110, or a memory card (for example, an SD card or a memory stick)installed in the portable terminal 100. The storage unit 175 may includea nonvolatile memory, a volatile memory, a Hard Disk Drive (HDD), or aSolid State Drive (SSD).

Further, the storage unit 175 can store applications having variousfunctions such as navigation, video phone communication, game, and alarmapplications based on the time, images for providing a Graphical UserInterface (GUI) related to the applications, databases or data relatedto a method of processing user information, a document, and a touchinput, and background images (a menu screen, an idle screen or the like)required for driving the portable terminal 100, operating programs, andimages photographed by the camera module 150.

In addition, the storage unit 175 can store data related to a combinedimage providing method in accordance with embodiments described herein.

The storage unit 175 is a machine (for example, computer)-readablemedium. The term “machine readable medium” refers to a medium providingdata to the machine such that the machine performs a specific function.The storage unit 175 can include a non-volatile medium and/or a volatilemedium. Both of these media are tangible so that commands transmittedthrough the media can be detected by a physical mechanism reading thecommands through the machine.

Examples of the machine readable medium include at least one of a floppydisk, a flexible disk, a hard disk, a magnetic tape, a Compact DiscRead-Only Memory (CD-ROM), an optical disk, a punch card, a paper tape,a RAM, a Programmable Read-Only Memory (PROM), an Erasable PROM (EPROM),and a flash-EPROM, but are not limited thereto.

The power supplier 180 can supply power to one battery or a plurality ofbatteries arranged at the housing of the portable terminal 100 accordingto a control of the controller 110. The one battery or the plurality ofbatteries supply power to the portable terminal 100. Further, the powersupplier 180 can supply power input from an external power source to theportable terminal 100 through a wired cable connected to the connector165. In addition, the power supplier 180 can supply power wirelesslyinput from the external power source to the portable terminal 100through a wireless charging technology.

Further, the portable terminal 100 includes at least one touch screen190 providing user graphic interfaces corresponding to various services(for example, phone communication, data transmission, broadcasting, andphotography) to the user.

The touch screen 190 can output an analog signal corresponding to atleast one user input received by the user graphic interface to the touchscreen controller 195.

The touch screen 190 can receive at least one user input through a bodyof the user (for example, fingers including a thumb) or the input unit168 (for example, a stylus pen or an electronic pen).

The touch screen 190 can receive successive motions of one touch (thatis, a drag input). The touch screen 190 can output an analog signalcorresponding to the successive motions of the input touch to the touchscreen controller 195.

The touch used in the present invention is not limited to a contactbetween the touch screen 190 and the finger or input unit 168, and mayinclude a noncontact interaction (for example, a case where the userinput means is located within a recognition distance (for example, 1 cm)where the user input means can be detected without a direct contact). Adistance or interval within which the user input means can be recognizedin the touch screen 190 may be designed according to a capacity orstructure of the portable terminal 100. Particularly, the touch screen190 is configured to output different values (for example, including avoltage value or a current value as an analog value) detected by adirect touch event and a hovering event so that the direct touch eventby a contact with the user input means and the direct touch event (thatis, the hovering event) can be distinguishably detected.

The touch screen 190 can be implemented in, for example, a resistivetype, a capacitive type, an infrared type, an acoustic wave type, or acombination thereof.

Further, the touch screen 190 includes two or more touch panels whichcan detect touches or approaches of the finger and the input unit 168 inorder to receive inputs by the finger and the input unit 168,respectively. The two or more touch panels provide different outputvalues to the touch screen controller 195. Then, the touch screencontroller 195 can recognize the different values input to the two ormore touch panels to distinguish whether the input from the touch screen190 is an input by the finger or an input by the input unit 168.

The touch screen controller 195 converts an analog signal input from thetouch screen 190 to a digital signal and transmits the converted digitalsignal to the controller 110. The controller 110 can control the touchscreen 190 by using the digital signal received from the touch screencontroller 195. For example, the controller 110 allows a short-cut icon(not shown) or an object displayed on the touch screen 190 to beexecuted in response to the direct touch event or the hovering event.Further, the touch screen controller 195 may be integrated with thecontroller 110.

The touch screen controller 195 can identify a hovering interval ordistance as well as a user input position by detecting a value (forexample, a current value or the like) output through the touch screen190, and convert the identified distance value to a digital signal (forexample, a Z coordinate) and then provide the converted digital signalto the controller 110. Further, the touch screen controller 195 incertain implementations can detect pressure with which the user inputmeans presses the touch screen 190 by detecting a value (for example, acurrent value or the like) output through the touch screen 190, andconverts the identified pressure value to a digital signal and thenprovide the converted digital signal to the controller 110.

FIG. 2 illustrates an example front perspective view of the portableterminal 100, and FIG. 3 illustrates an example rear perspective view ofthe portable terminal. As shown in FIGS. 2 and 3, the touch screen 190is centrally disposed on a front surface 101 of the portable terminal100, and preferably occupies most of the front surface 101. FIG. 2 showsan example where a main home screen is displayed on the touch screen190. The main home screen is a first screen displayed on the touchscreen 190 when power of the portable terminal 100 is turned on.Further, when portable terminal 100 has different home screens ofseveral pages, the main home screen may be a first home screen of thehome screens of several pages. Short-cut icons 191-1, 191-2, and 191-3for executing frequently used applications, a main menu switching key191-4, time, weather and the like may be displayed on the home screen.When the user selects the main menu switching key 191-4, the menu screenis displayed on the touch screen 190. Further, a status bar 192 whichdisplays the status of the portable terminal 100 such as a batterycharging status, a received signal intensity, and a current time may beformed on an upper end of the touch screen 190.

A home button 161 a, a menu button 161 b, and a back button 161 c may beformed on a lower end of the touch screen 190.

The home button 161 a displays the main home screen on the touch screen190. For example, when the home button 161 a is selected in a statewhere a home screen different from the main home screen or the menuscreen is displayed on the touch screen 190, the main home screen isdisplayed on the touch screen 190. Further, when the home button 161 ais selected while applications are executed on the touch screen 190, themain home screen shown in FIG. 2 is displayed on the touch screen 190.In addition, the home button 161 a may be used to display recently usedapplications or a task manager on the touch screen 190.

The menu button 161 b provides a connection menu which can be displayedon the touch screen 190. The connection menu includes a widget additionmenu, a background changing menu, a search menu, an editing menu, anenvironment setup menu and the like.

The back button 161 c can be used for displaying the screen which wasexecuted just before the currently executed screen or terminating themost recently used application.

The first camera 151, the luminance sensor 170 a, and the proximitysensor 170 b may be disposed on edges of the front surface 100 a of theportable terminal 100. The second camera 152, the flash 153, and thespeaker 163 may be disposed on a rear surface 103 of the portableterminal 100.

For example, a power/reset button 161 d, volume buttons 161 e having avolume increase button 161 f and a volume decrease button 161 g, aterrestrial DMB antenna 141 a for broadcasting reception, and one or aplurality of microphones 162 may be disposed on a side surface 102 ofthe portable terminal 100. The DMB antenna 141 a may be fixed to theportable terminal 100 or may be formed to be detachable from theportable terminal 100.

Further, the connector 165 is formed on a side surface of a lower end ofthe portable terminal 100. A plurality of electrodes are formed in theconnector 165, and the connector 165 can be connected to the externaldevice through a wire. The earphone jack 167 may be formed on a sidesurface of an upper end of the portable terminal 100. Earphones may beinserted into the earphone jack 167.

Further, the input unit 168 may be mounted to a side surface of a lowerend of the portable terminal 100. The input unit 168 can be insertedinto the portable terminal 100 to be stored in the portable terminal100, and withdrawn and separated from the portable terminal 100 when itis used.

Meanwhile, although FIG. 1 illustrates the two cameras 151 and 152, twoor more cameras can be mounted at different positions. Accordingly, thepresent invention can be similarly applied to the process of combiningimages photographed through a plurality of cameras at positions otherthan front and rear camera positions to make the images one combinedimage. That is, the one combined image can be generated by selecting oneimage from images photographed through the first camera 151, selectingone image from images photographed through the second camera 152,selecting one image from images photographed through a third camera (notshown), and combining the selected images. However, a case of using thefirst camera 151 and the second camera 152 will be described as anexample for convenience of the description hereinafter.

An embodiment of the present invention provides an image generated bycombining a rear view image photographed by the second camera 152 and afront view image photographed by the first camera 151. Hereafter, forconvenience of description of this embodiment, the first and secondcameras will be interchangeably called the front and rear cameras,respectively. The rear is a direction which projects from a rear surfaceof the portable terminal 100, and the front is the opposite directionwhich projects from a front surface of the portable terminal. Thus thefront camera 151 can be constructed to photograph a user's face or body,and the rear camera 152 can be constructed to photograph a subject to bephotographed or a foreground in a direction viewed from the user's eyes.

The controller 110 controls general operations of the portable terminal100 and controls other components within the portable terminal 100 toperform a combined image providing method. Particularly, the controller110 analyzes front view images and rear view images acquired byphotographing the user and the subject by controlling a plurality ofcameras mounted to the portable terminal 100, selects effective imagesfrom the front view images and the rear view images, and then performs afunction of controlling an operation for combining the effective imagesselected from the front view images and rear view images. In theembodiment of the present invention, the operation of analyzing theimages refers to an operation of analyzing whether there are effectivefront and rear view images which satisfy a classification referencethrough application of the classification reference to front and rearview images received through respective cameras.

Prior to describing further details of certain embodiments of thepresent invention, a “dual shot” and a “best shot” used in suchembodiments will be briefly described.

A “dual shot” corresponds to a function of simultaneously orsequentially photographing two images by using cameras located indifferent positions and combining the two images into one image. A “bestshot” corresponds to a function of successively photographing aplurality of images for the purpose of capturing a still image, andselecting one of the images as a best image according to a predeterminedclassification reference. Here, the predetermined classificationreference is a numerical value of a photography metric such as a bluramount within the image, a facial expression of the person, a shootingcomposition, and the like.

When simultaneous or staggered photographing is performed through thefront camera 151 and the rear camera 152 in a dual shot mode, qualitydegradation may occur in at least one of the front and rear view images.For example, when the user photographs the subject by using the rearcamera 152, the user should adjust a direction of the rear camera 152such that the subject enters an angle of view. At this time, imageblurring by hand trembling and/or a shooting composition such as levelmaintenance (i.e., avoiding undesired tilting) should be alsoconsidered. At the same time, the user should pay attention topositioning his/her own face in a desired position as shown on thedisplay and being photographed without blinking through the front camera151. Thus, in the dual shot mode, since satisfaction of the actuallyphotographed images may deteriorate due to various photographingconditions which are more complex than in conventional, single cameraphotography, quality degradation may occur when the photographed imagesare combined. Accordingly, embodiments of the present invention providea method of guaranteeing quality of the combined image by combining onlyimages which satisfy a predetermined classification reference throughanalysis of images photographed in the dual shot mode.

FIG. 4 is an operation flowchart for an image combination methodaccording to a first embodiment of the present invention. The followingdescription of FIG. 4 will be made with reference to FIGS. 5, 6A and 6Bto facilitate an understanding of the method.

When a camera application is executed, the controller 110 outputs firstand second preview images received in real time through the front andrear cameras, respectively to the touch screen 190 in step 400.Accordingly, the user can preview a front view image received throughthe front camera, for example, an image of the user and preview a rearview image received through the rear camera, for example, an image ofthe external subject. As described above, the user can determine when toperform a photographing action through the screen outputting the previewimages. Here, the screen outputting the preview images can be configuredin a PIP form, or the preview images received through the first andsecond cameras, for example, front and rear view preview images can bedisplayed in a position and a size designated by the user. In a statewhere the preview images are output, when a photographing signal isinput in step 405, the controller 110 stores a predetermined number offront and rear view images each successively photographed through thefirst and second cameras in a buffer in step 410. Herein, “photographingsignal” refers to a command signal generated by the portable terminal100 responsive to a user input command for the capture of an image bythe respective camera, or by both cameras. In one implementation, asingle shutter icon or physical shutter key (if provided) is displayedto enable capture of both front and rear images simultaneously. Inanother implementation, two shutter icons, one for each camera, aredisplayed concurrently.

FIG. 5 illustrates images A, B, C, D, E, and F successively photographedat a predefined time interval through the second camera and images 1, 2,3, 4, 5, and 6 successively photographed through the first camera. In anembodiment, the predetermined number of successively photographed frontand rear view images are sequentially stored in the buffer before andafter the photographing signal is input as illustrated in FIG. 5. It isnoted, to capture the images just before the photographing signal isinput, the portable terminal continually captures and displays previewimages, and temporarily stores a minimum number of the most recentpreview images in a buffer. In this manner, as soon as the user commandsthe photographing signal, the most recent stored preview images are usedfor the first portion of the successive images, e.g., for the A, B and Cor 1, 2 and 3 images in FIG. 5.

Subsequently, the controller 110 analyzes whether there are front andrear view images which satisfy a predetermined classification referencein the front and rear view images successively photographed through thefirst and second cameras in step 415. Here, the predetermined referenceincludes at least one of a blur amount within the image, a specificpattern of a person, and a shooting composition, and corresponds to anumerical value thereof. Further, the same classification reference maybe applied to each of the front and rear view images, but it is apparentthat different classification references may be applied to the front andrear view images according to characteristics of the front and rear viewimages. For example, the front view image may be analyzed based on thespecific pattern of the person since the front view image is a close-upimage generated by photographing the user, and the rear view image maybe analyzed based on the blur amount since the rear view image is animage generated by photographing a typical external subject located asignificant distance away, such as a scenic image.

Now, the blur amount of the predetermined classification referenceaccording to an embodiment of the present invention can be a numericalvalue representing image contrast. That is, the blur amount can beestablished as a numerical value of a hand trembling degree of the user,a motion degree of the subject within an angle of view, and/or a degreeby which the camera is out of focus (e.g., due to improper focus). Thenumerical value may be set based on pre-learned or predetermined setinformation. Accordingly, the controller 110 determines the blur amountof each of the front and rear view images when the images are analyzed.Suitable algorithms to determine blurring based on image and motionanalysis are known by those of skill in the art, thus descriptionthereof is omitted. Further, the specific pattern of the person of thepredetermined classification reference can be at least one of aparticular facial expression (laughing, frightened and the like),whether eyes are closed, whether the person looks straight ahead, andwhether a red eye effect is generated, and can be expressed by anumerical value thereof. Specifically, for laughing criteria, in aportrait image generated by photographing the external subject or theuser, a laughing degree is compared with a learning model input inadvance for each level of laughing, and a laughing level of the learningmodel having the highest similarity with the laughing degree isdetermined. In conjunction with the laughing parameter, whether the eyesare closed, the red eye effect is generated, and the face looks forwardare determined as weights and thus a facial expression score becomes anumerical value including the determined laughing level and the weights.

In addition, the shooting composition of the predeterminedclassification reference can be established as a numerical value of amatching degree between locations of the user's face and the subject bycomparing the locations in shooting compositions divided into halves andthirds. The appropriateness of the shooting composition determineswhether the shooting composition is the best shooting composition tosimultaneously photograph the user and the external subject, that is,whether the shooting composition determined through the image analysisis the best shooting composition. Here, the shooting composition isacquired by using saliency information through which a feature pointwithin the image and factors such as a position and a size of the facecan be detected.

By applying the numerical values for the classification reference to theanalyzed images, the controller 110 determines whether there are frontand rear view images which satisfy the classification reference criteriain the successively photographed front and rear view images in step 420.

For example, when the blur amount of the predetermined classificationreference is used, the controller 110 determines whether the blur amountof each of the successively photographed front and rear view images isequal to or lower than a threshold. If so, the corresponding image isdetermined as the image which satisfies the classification reference.Since there are a plurality of successively photographed images taken byeach camera, a plurality of images which satisfy the classificationreference may exist. In this case, an image including the smallest bluramount is selected as a best image from the images having the bluramount equal to or lower than the threshold.

Further, when the specific pattern of the person of the predeterminedclassification reference is applied, a value indicating a specificpattern of the person for each of the successively photographed frontand/or rear view images is calculated. By applying the classificationreference corresponding to the specific pattern of the person, apriority of each of the front and rear view images can be set. When thespecific pattern of the person is applied, a best front or rear viewimage can be selected through comparison with the threshold.

Further, when the shooting composition of the predeterminedclassification reference is applied, a numerical value of the matchingdegree between the position of the user's face and the position of theexternal subject within the successively photographed front and rearview images generated by comparing the positions in shootingcompositions divided into halves and thirds is compared with thethreshold. By applying the classification reference corresponding to theshooting composition, a priority of each of the front and rear viewimages can be set. When the shooting composition is applied, a bestfront or rear view image can be selected through comparison with thethreshold. For example, in a succession of front camera images of theuser's face, an image that is the most centered might be selected as thebest front image with respect to the shooting composition parameter.Similarly, in a rear image that includes a person, an image containing asubject's entire head would be selected over ones with heads “cut off”in the image. Predetermined composition criteria can be used for scenictype images as well. For instance, in a photograph of a building, animage capturing the entire top or bottom portion of a building might beautomatically selected over one in which the top or bottom portion iscut off.

Alternatively, instead of applying only a single parameter for theclassification reference, two or more of the blur amount, the specificpattern of the person, and the shooting composition parameters may beapplied to the successively photographed front and/or rear view images.In this case, one front view image and one rear view image having themost similar value with a reference value can be selected. In otherwords, if no single image meets the criteria of all the differentparameters, an image that comes closest to satisfying all the criteriamay be selected.

When there is one front view image and one rear view image which satisfythe classification reference, the controller 110 selects these images instep 425, combines the two selected images and then stores the combinedimage in the storage unit 175 in step 430. For example, in FIG. 5, whenthe rear view image which satisfies a first classification reference isthe image B among the images A, B, C, D, E, and F successivelyphotographed through the second camera and the front image whichsatisfies a second classification reference is the image 3 among theimages 1, 2, 3, 4, 5, and 6 successively photographed through the firstcamera, one combined image 500 is generated by combining the images Band 3.

When there is no front and rear view image which satisfies theclassification reference in step 420, for example, when it is determinedthat there is no front view image and/or rear view image which satisfyat least a minimum individual parameter or combined parameter thresholdof the classification reference among the front view images and the rearview images, the process returns to step 400.

For example, in order to acquire the best front and rear view images inthe dual shot mode, the desired external subject view, the portableterminal 100 and the user should be located on a straight line. However,for example, as shown in FIG. 6A, when the external subject 610, theportable terminal 100, and the user 620 cannot be located on thestraight line because of an obstacle 615, that is, when there is anobstacle at a lower side between the external subject and the portableterminal 100, the user raises the portable terminal 100 to avoid theobstacle and then photographs the external subject 610. Similarly, theportable terminal 100 should be controlled in a low position in close-upphotography, for example, during a close-up photo of a friend using therear camera for the purpose of creating a combined image of the user andher friend. In this case, the external subject, the portable terminal100, and the user have a difficulty in being located on the straightline.

Further, when the photographing is performed in outdoor bright light, ascreen may not be viewed well. For example, when visibility is low dueto a backlight or a light reflection, better visibility can be obtainedby changing an angle of a display of the portable terminal 100. However,such a motion is impossible in an ideal traditional dual shot, since thefront image no longer ideally captures an image of the user's face dueto tilt of the portable terminal to implement the angle change. Due tothe tilt, the photographing may only be possible without the userwatching the preview. In this case, a user's facial expression may beunnatural or a composition such as a position of the face within anangle of view may be different from an intention. Since the externalsubject and the user's face are not normally included in the front andrear view images photographed in such a state, it is determined that theshooting composition is improper. Accordingly, the controller 110determines that there is no front or rear view image which satisfies theclassification reference.

In this case, the external subject 610 is located with a photographingangle of view 600 of the rear camera as illustrated in FIG. 6A, so thata best shot of the external subject 610 can be acquired based on avisibility or other criteria (visibility may be considered an aspect ofthe blurring criteria described above). Accordingly, the rear view imagewhich satisfies the classification reference can be acquired, but thefront view image cannot satisfy the classification reference. Therefore,the controller 110 displays a front preview image received through thefront camera by returning to step 400 to drive again only the frontcamera and then analyzes front view images successively photographedthrough the front camera when a photographing signal is input in step405. During this process, controller 110 may display prompts instructingthe user, or output audio prompts, to advise the user to restore aposition of the camera for capturing a best front view shot andcompleting a satisfactory PIP combined image using the already capturedbest rear view shot.

For example, when the best shot of the external subject 610 is acquiredthrough the second camera as illustrated in FIG. 6A and then thephotographing is performed after moving the portable terminal 100 asillustrated in FIG. 6B, a best shot of the user 620 located within aphotographing range 605 can be acquired even though there is theobstacle 615 within the photographing range 600 of the second camera.Alternatively, when the user moves the portable terminal 100 tosuccessively photograph images with various shooting compositions duringthe photographing, the controller 110 can recommend, through a displayprompt or the like, a best shooting composition through the imageanalysis. As described above, according to the present invention, eventhough the external subject, the portable terminal 100, and the user arelocated on the straight line, it is possible to acquire the combinedimage photographed with the best shooting composition.

As described above, when at least one of the front and rear view imagesdoes not satisfy the classification reference in step 420, thecontroller 110 controls an operation for re-photographing. Accordingly,the process returns to step 400. Further, when both the front and rearview images do not satisfy the classification reference, the front andrear view images are photographed again by driving both the first andsecond cameras. When only one of the front and rear view images does notsatisfy the classification reference, only one camera for photographingthe corresponding image is driven. For example, when there is an imagewhich satisfies the classification reference among the front viewimages, the front view image which satisfies the classificationreference is stored and then re-photographing is performed to acquire arear view image which satisfies the classification reference. When thereis a selected rear view image which satisfies the classificationreference among the re-photographed rear view images, the rear viewimage is combined with the pre-stored front view image and then thecombined image is stored.

Turning now to FIG. 7, an operation flowchart illustrating an imagecombining process according to a second embodiment of the presentinvention is shown. This process performs single image photographingthrough one of the first and second cameras and successive photographingthrough the other camera, combining an image corresponding to the bestshot among the successively photographed images and the imagephotographed only once, and storing the combined image.

Steps 700 and 705 are identical to steps 400 and 405 of FIG. 4, thusdescription thereof is omitted. Once a photographing signal is input atstep 705, in step 710 a first image photographed through the firstcamera is stored in a buffer, without capturing and analyzing aplurality of successive first images, while successive images arephotographed through the second camera and stored in the buffer. Next,in step 715, it is analyzed whether there is an image which satisfiesthe classification reference in the successively photographed images.(Note that the roles of the first and second cameras can be reversed viasuitable selection in a settings menu or by preset design, such thatsingle image photographing is performed through the second camera andsuccessive photographing is performed through the first camera in anidentical process to that just described.) Subsequently, it isdetermined whether there is an image which satisfies the classificationreference in the successively photographed images in step 720, a secondimage which satisfies the classification reference is selected in step725, and the first image and the second image are combined and thecombined images are stored in step 730.

FIG. 8 illustrates an image A photographed in a single image captureoperation through the first camera and images 1, 2, 3, 4, 5, and 6photographed through the second camera. The single image and thepredetermined number of successively photographed images aresequentially stored in the buffer before and after the photographingsignal is input as illustrated in FIG. 8. In the example shown, when animage which satisfies the classification reference is the image 3 amongthe images 1, 2, 3, 4, 5, and 6 successively photographed through thesecond camera, one combined image 800 is generated by combining theimage A photographed through the first camera and the image 3.

FIG. 9 is an operation flowchart illustrating an image combining processaccording to a third embodiment of the present invention, andillustrates a process of controlling conditions for image processingbefore photographing front and rear view images to minimize an effect bya photographing environment and then selecting images to be combined.

In a state where first and second preview images received through thefirst and second cameras, respectively, are output in step 900, thecontroller 110 at step 905 analyzes brightness and tone of the first andsecond preview images. The analysis is for determining a state of thephotographing environment. Subsequently, it is determined whetherdifferences between brightness and/or tone of the first preview image ascompared to second preview image are equal to or larger than a threshold(or respective thresholds for brightness and tone) in step 910. Here,the brightness comparison corresponds to an operation of comparingbrightness when a sensor gain and an exposure time of at least one ofthe first and second cameras are restricted as a value equal to or lowerthan a predetermined level. Further, the tone comparison corresponds toan operation of comparing at least one of a dynamic range within theimage or a histogram shape.

In the case where the difference is equal to or larger than thethreshold, when the first and second preview images are combined, thedifferences may be very obvious, so that the controller 110 performs anoperation of controlling an exposure condition to make brightness andtone between the first and second preview images similar in step 915.When the difference is lower than the threshold, brightness and tonebetween the first and second preview images are similar, so that anatural combined image can be acquired when the images are combined.

In an embodiment, a method of controlling the exposure conditionincludes exposure correction modes such as a High Dynamic Range (HDR)and a Low Light Shot (LLS). For example, when a gradation difference ofthe front view image is larger than a gradation difference of the rearview image, the exposure condition of the HDR is set as (+1,0,−1) forthe front view image such that a difference in steps according to anexposure chart is small, and set as (+2,0,−2) for the rear view imagesuch that a difference in steps according to an exposure chart is large.Further, when brightness of the front view image is brighter than therear view image, a sensor gain for the front view image may be set low,for example, ISO 100, and a sensor gain for the rear view image may beset relatively high, for example, ISO 200 in the LLS exposure.

As described above, in order to reduce the differences in brightness andtone between the front view image and the rear view image due to theeffect of the surrounding photographing environment, the process ofcontrolling the exposure condition to make similar brightness and toneis performed before taking an actual picture after comparing differencesin brightness and tone between the front view image and the rear viewimage received before the photographing. Then, when the photographingsignal is input in step 920, the controller 110 selects the first imagephotographed through the first camera and the second image photographedthrough the second camera in step 925, and then combines the first andsecond images and stores the combined images in step 930.

FIG. 10 illustrates an example of combining images in accordance withthe method of FIG. 9. Images 1-1, 1-2, and 1-3 photographed through thefirst camera according to the HDR exposure correction mode and stored inthe buffer become one HDR image 1 and images 2-1, 2-2, and 2-3photographed through the second camera and stored in the buffer becomeanother HDR image 2. Then, one combined image 1000 is generated bycombining the HDR image 1 and the HDR image 2 corresponding to the frontand rear view images. Further, in the LLS exposure correction mode, anLLS front view image and an LLS rear view image become one combinedimage. Note that while FIG. 10 illustrates the case of applying the HDRexposure correction mode to both the front and rear view images, the HDRexposure correction mode can be applied to only one image, while stillachieving some improvement in harmonizing the front and rear images.

When respective effective images are selected from the imagesphotographed under the exposure condition controlled in the above way,the best combined image can be acquired. For example, when a subjectphotographed through the first camera is a person, the best combinedimage can be acquired by segmenting the person and combining thesegmented image and the image photographed through the second camera.Although the exposure correction has been described as an example,operations such as color correction, white balance and the like may beperformed alternatively or additionally.

FIG. 11 is an operation flowchart illustrating an image combiningprocess according to a fourth embodiment of the present invention. Thisprocess generates a combined image by selecting effective images fromimages photographed through the respective cameras when one of the firstand second cameras 151 and 152 is operated in the best shot mode and theother camera is operated in the exposure correction mode.

Referring to FIG. 11, detailed descriptions of steps 1100 to 1120 isomitted since the steps are identical to operations of steps 900 to 920of FIG. 9. However, in FIG. 11, when the photographing signal is input,the first image (which can be either a front or rear image)corresponding to a predetermined classification reference is selectedfrom images successively photographed through the first camera in step1125. Further, the second image (which can be either a rear or frontimage) photographed through the second camera in the exposure correctionmode is selected in step 1130. Subsequently, the first image and thesecond image are combined and then the combined images are stored instep 1135.

FIG. 12 illustrates images A, B, C, D, E, and F successivelyphotographed through the first camera operating in the best shot modeand stored in the buffer and images 1-1, 1-2, and 1-3 photographedthrough the second camera operating in the exposure correction mode andstored in the buffer. When the images 1-1, 1-2, and 1-3 stored in thebuffer become the one HDR image 1 and an image which satisfies theclassification reference is the image B among the images A, B, C, D, E,and F through the image analysis according to the best shot mode, theimage B corresponding to the best shot through the first camera and theexposure-corrected HDR image 1 are combined and then one combined image1200 is generated.

Although exemplary embodiments of the present invention have beendescribed, the present invention is not limited the above describedspecific embodiments. For example, there is a method of combining imagesinto one image by considering a matching degree between the front andrear view images. An example of the method of considering the matchingdegree includes a method of selecting an image to be combined based onsimilarity between facial expressions of the photographer and the personcorresponding to the external subject which includes a process ofcomparing a laughing level of the person corresponding to the subjectwith an image for each pre-modeled laughing level, classifies thelaughing level of the subject as a level having a highest similarity,and selecting, when a laughing level of the photographer is most similarto the laughing level of the subject, the corresponding images as imagesto be combined.

Another example considering the matching degree includes a method ofrecognizing the photographer and the subject corresponding to the personthrough a face recognition and applying different placements accordingto a relation therebetween when they are combined. There is a method ofdisposing an image of the photographer close to a subject having aclosest relation with the photographer when there are a plurality ofsubjects. Here, the relation can be designated as a collective numericalvalue of the number of times of phone calls, call duration, the numberof times of message transmission, and the number of times of SNSconfirmation. Other examples of the method considering the matchingdegree includes a method of comparing face sizes of the photographer andthe subject corresponding to the person, resizing the face sizes to besimilar through e.g., the use of different zoom settings, and thenplacing the images of the photographer and subject having the resizedface in a predetermined relationship in the composite image. Yet anothermethod corrects for a condition of an atmosphere difference between thephotographer and the subject due to a lighting condition or a screenbackground color, by unifying atmospheres of front/rear images throughchanging a color tone and setting an exposure condition suitable for thecorresponding tone.

When the user desires to change a combination of best images recommendedto the user by the method (through display prompts, etc.) consideringthe matching degree, the combination may be changed in the followingway. For example, when the user desires to change the combination offront/rear view images manually, there is a method of sequentiallydisposing candidate images according to the matching degree and settinga type of combination differently based on the user's selection.Alternatively, there is a method of performing a switching to the nextcandidate image when selecting the front/rear view image from thecombined image without displaying candidate images in the manualcombination. In another alternative method, when the front view image ischanged, the rear view image is automatically changed to one having thehighest matching degree.

In the above examples, the first camera has been generally described inthe context of the front camera, and the second camera has beengenerally described in the context of the rear camera; however, thisrelationship may be interchanged as desired in other applications. Forinstance, the user may desire that the front image is the larger imageand the rear image is the reduced image in a PIP combination. In someembodiments, the PIP front and rear image relationship, i.e., whichimage is the larger image and which is the smaller image, is settable ina settings menu or the like.

Further, although the touch screen has been described as arepresentative example of the display unit for displaying the screen,general display units such as a Liquid Crystal Display (LCD), an OrganicLight Emitting Diode (OLED), an LED and the like which do not have atouch detection function may be used instead of the touch screen.

According to embodiments of the present invention, when different imagesare combined by using a plurality of cameras, there is an effect ofimproving quality of a combined image by combining images which satisfya predetermined reference, and there is an advantage of increasingconvenience in using a combination function through the predeterminedreference.

Further, embodiments have an advantage in that the user can use an imagecombining function even when it is difficult to simultaneously securetwo or more shooting compositions by photographing an image through oneof the front and rear cameras and then photographing a differentdirection image through the other camera.

In addition, embodiments have an effect of reducing an obviousness ofdifference of the combined image by performing the photographing aftercontrolling brightness and tone of images to be combined. Furthermore,the present invention improves convenience in re-photographing an imageby, when a quality of only one of the images photographed in a dual shotmode is equal to or lower than a reference value, storing the otherimage and then combining the other image with the re-photographed image.

It may be appreciated that embodiments of the present invention can beimplemented in software, hardware, or a combination thereof. Any suchsoftware may be stored, for example, in a volatile or non-volatilestorage device such as a ROM, a memory such as a RAM, a memory chip, amemory device, or a memory IC, or a recordable optical or magneticmedium such as a CD, a DVD, a magnetic disk, or a magnetic tape,regardless of its ability to be erased or its ability to be re-recorded.It can be also appreciated that the memory included in the mobileterminal is one example of machine-readable devices suitable for storinga program including instructions that are executed by a processor deviceto thereby implement embodiments of the present invention. Therefore,embodiments of the present invention provide a program including codesfor implementing a system or method claimed in any claim of theaccompanying claims and a machine-readable device for storing such aprogram. Further, this program may be electronically conveyed throughany medium such as a communication signal transferred via a wired orwireless connection, and embodiments of the present inventionappropriately include equivalents thereto. Any of the functions andsteps provided in the Figures may be implemented in hardware, softwareor a combination of both and may be performed in whole or in part withinthe programmed instructions of a computer. No claim element herein is tobe construed under the provisions of 35 U.S.C. 112, sixth paragraph,unless the element is expressly recited using the phrase “means for”.

Further, the portable terminal can receive the program from a programproviding apparatus connected to the portable terminal wirelessly orthrough a wire and store the received program. The program providingapparatus may include a memory for storing a program containinginstructions for allowing the portable terminal to perform a presetcontent protecting method and information required for the contentprotecting method, a communication unit for performing wired or wirelesscommunication with the portable terminal, and a controller fortransmitting the corresponding program to the portable terminalaccording to a request of the portable terminal or automatically.

While the present invention has been shown and described with referenceto certain embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present invention asdefined by the appended claims.

What is claimed is:
 1. A method of combining effective images in anelectronic device having a plurality of cameras, the method comprising:in response to a photographing signal, temporarily storing imagessuccessively photographed through a first camera; selecting a firstimage which satisfies a predetermined classification reference from theimages successively photographed through the first camera; and combininga second image photographed through a second camera with the firstimage.
 2. The method of claim 1, wherein the predeterminedclassification reference is a first classification reference, and themethod further comprising: temporarily storing images successivelyphotographed through the second camera in response to the photographingsignal; and selecting the second image as an image that satisfies asecond predetermined classification reference from the successivelyphotographed images, wherein the selected second image and the firstimage are combined.
 3. The method of claim 2, further comprising:re-photographing images through at least one of the first camera and thesecond camera when there is no image that satisfies the first or secondclassification reference in the images successively photographed throughthe first and second cameras; and determining whether there is an imagewhich satisfies the classification reference in the images successivelyre-photographed through the first camera or the second camera.
 4. Themethod of claim 1, wherein the predetermined classification reference isa numerical value of at least one of a blur amount within an image, aspecific pattern of a person, and a shooting composition.
 5. The methodof claim 4, wherein the blur amount is a numerical value of a contrastsize of the image.
 6. The method of claim 4, wherein the specificpattern of the person is a numerical value using a detection of at leastone of a specific facial expression of the person, a closed eyescondition, a straight look ahead condition, and a red eye effect.
 7. Themethod of claim 4, wherein the shooting composition is a numerical valueof a matching degree between a position of a user's face and a positionof a subject in a shooting composition divided into halves or thirds bycomparing the positions.
 8. The method of claim 1, further comprising:previewing the images received through the first camera and the secondcamera; analyzing a difference in at least one of brightness and tonebetween the previewed images; and controlling an exposure condition whenthe difference between the previewed images is equal to or larger than athreshold.
 9. The method of claim 8, wherein the images photographedthrough the first camera and the second camera which operate inaccordance with the controlled exposure condition are combined.
 10. Anelectronic device having a plurality of cameras for combining effectiveimages, the electronic device comprising: a first camera whichsuccessively photographs images in response to a photographing signal; asecond camera which photographs a second image; and a controller whichselects a first image satisfying a predetermined classificationreference from the images successively photographed through the firstcamera and combines the first and second images.
 11. The electronicdevice of claim 10, wherein the predetermined classification referenceis a first classification reference, and the controller temporarilystores images successively photographed through the second camera inresponse to the photographing signal, selects the second image whichsatisfies a second predetermined classification reference from thesuccessively photographed images, combines the selected second image andthe first image, and then stores the combined image in a storage unit.12. The electronic device of claim 11, wherein the controllerre-photographs images through at least one of the first camera and thesecond camera when there is no image which satisfies the classificationreference in the images successively photographed through the first andsecond cameras, and determines whether there is the image whichsatisfies the classification reference in the images successivelyre-photographed through the first or second cameras.
 13. The electronicdevice of claim 10, wherein the predetermined classification referenceis a numerical value of at least one of a blur amount within an image, aspecific pattern of a person, and a shooting composition.
 14. Theelectronic device of claim 13, wherein the blur amount is a numericalvalue of a contrast size of the image, the specific pattern of theperson is a numerical value using at least one of a detection of aspecific facial expression of the person, a closed eyes condition, astraight look ahead condition, and a red eye effect, and the shootingcomposition is a numerical value of a matching degree between a positionof a user's face and a position of a subject in a shooting compositiondivided into halves or thirds by comparing the positions.
 15. Theelectronic device of claim 10, further comprising a display unit whichpreviews the images received through the first camera and the secondcamera.
 16. The electronic device of claim 15, wherein the controlleranalyzes a difference in at least one of brightness and tone between thepreviewed images, and controls an exposure condition when the differencebetween the previewed images is equal to or larger than a threshold. 17.The electronic device of claim 16, wherein at least one of the imagesphotographed through the first camera and the second camera correspondsto one image of a High Dynamic Range (HDR) and a Low Light Shot (LLS)according to the controller exposure condition.
 18. The electronicdevice of claim 10, wherein the images are combined in apicture-in-picture (PIP) type fashion.
 19. A method of combiningeffective images in an electronic device having front and rear cameras,the method comprising: in response to a photographing signal, capturingfront and rear images from the front and rear cameras, respectively;determining a difference in at least one of brightness and tone betweenthe front and rear images; modifying at least one of the front and rearimages to equalize brightness or tone between the front and rear imagesif the difference exceeds a predetermined threshold; and combining thefront and rear images.
 20. A non-transitory computer-readable storagemedium having stored therein program instructions, which when executedby a computer, perform the method of claim 1.